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Abstract 

Background: Scoping studies are increasingly common for broadly searching the literature on a specific topic, yet 
researchers lack an agreed-upon definition of and framework for the methodology. In 2005, Arksey and O'Malley 
offered a methodological framework for conducting scoping studies. In their subsequent work, Levac et al. 
responded to Arksey and O'Malley's call for advances to their framework. Our paper builds on this collective work 
to further enhance the methodology. 

Discussion: This paper begins with a background on what constitutes a scoping study, followed by a discussion 
about four primary subjects: (1) the types of questions for which Arksey and O'Malley's framework is most 
appropriate, (2) a contribution to the discussion aimed at enhancing the six steps of Arskey and O'Malley's 
framework, (3) the strengths and challenges of our experience working with Arksey and O'Malley's framework as a 
large, inter-professional team, and (4) lessons learned. Our goal in this paper is to add to the discussion encouraged 
by Arksey and O'Malley to further enhance this methodology. 

Summary: Performing a scoping study using Arksey and O'Malley's framework was a valuable process for our 
research team even if how it was useful was unexpected. Based on our experience, we recommend researchers be 
aware of their expectations for how Arksey and O'Malley's framework might be useful in relation to their research 
question, and remain flexible to clarify concepts and to revise the research question as the team becomes familiar 
with the literature. Questions portraying comparisons such as between interventions, programs, or approaches 
seem to be the most suitable to scoping studies. We also suggest assessing the quality of studies and conducting a 
trial of the method before fully embarking on the charting process in order to ensure consistency. The benefits of 
engaging a large, inter-professional team such as ours throughout every stage of Arksey and O'Malley's framework 
far exceed the challenges and we recommend researchers consider the value of such a team. The strengths include 
breadth and depth of knowledge each team member brings to the study and time efficiencies. In our experience, 
the most significant challenges presented to our team were those related to consensus and resource limitations. 
Effective communication is key to the success of a large group. We propose that by clarifying the framework, the 
purposes of scoping studies are attainable and the definition is enriched. 
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Background 

Our research team, which is large and comprised of 
many disciplines, undertook a scoping study to investi- 
gate the information needs of people with colorectal 
cancer. We drew on the methods laid out in a 2005 
framework by Arksey and O'Malley [1] who were among 
the first scholars to articulate a framework to clarify the 
usefulness of and methods inherent in a scoping study. 
They claim an interest in stimulating further discussion 
on the value of scoping studies and encourage others to 
further develop the methodology. Since their framework 
was published, a handful of researchers have taken up 
their challenge [2-8]. Levac et al. published the most 
prominent of these in 2010, addressing the framework's 
strengths and limitations with the aim to encourage con- 
sistent methods across all scoping studies [8]. After 
reviewing our own research process and sharing our ex- 
periences with this methodology in this paper, we also 
contribute to the discussion that enhances Arksey and 
O'Malley's framework. We offer a unique perspective on 
this framework because of our collective experience as a 
large, inter-professional team. We also propose that the 
definition, purposes, and methodological framework of 
scoping studies cannot be clarified in isolation but must 
be amended in tandem. We offer recommendations that, 
when added to those of other researchers such as Levac 
et al., generate a strengthened definition and position 
the purposes as attainable. Interestingly, some of our cri- 
tiques and suggestions for the development of the meth- 
odology mirror those expressed by Levac et al. Since 
Levac et al's paper was published while our team was 
concluding our scoping study; we reached our conclu- 
sions independently of their suggestions. The fact that 
we share some similar conclusions indicates that the 
collective suggestions from Levac's team and ours are 
worth serious consideration for enhancing Arksey and 
O'Malley's framework. We also offer some proposals for 
consideration not yet expressed by others engaged in 
scoping studies, primarily drawn from our experience as 
a large, inter-professional team. Our team of twelve 
people included academic researchers as well as clinical 
practitioners working in a cancer treatment centre from 
a range of disciplines and practices, including nutrition, 
radiation therapy, patient and family clinical counseling, 
nursing, biology, library and information science, and 
the social sciences. Each member of our team shared a 
common interest in colorectal cancer and a desire to im- 
prove the quality of care received by people with this 
diagnosis. 

What is a scoping study? 

Over the last two decades, scoping studies have become 
an increasingly common method of searching the litera- 
ture on a specific topic. There is no one agreed upon 



definition for scoping studies, but there has been some 
effort among researchers to seek clarification. Levac 
et al. confirm this view in their work, pointing to seven 
recent authoritative scoping sources, each offering a dif- 
ferent definition or purpose. In spite of the varying inter- 
pretations in this methodology's definition, and because 
we used the original definition as laid out by Arksey and 
O'Malley in our research, we refer to the Arksey and 
O'Malley definition throughout this paper. 

Arskey and O'Malley begin with Mays et al's definition 
of scoping studies: "to map rapidly the key concepts 
underpinning a research area and the main sources and 
types of evidence available, and can be undertaken as 
standalone projects in their own right, especially where 
an area is complex or has not been reviewed compre- 
hensively before" [9]. The framework they developed 
serves to expand on this definition by identifying four 
main reasons for conducting a scoping study: (1) to 
examine the extent, range and nature of research activ- 
ity; (2) to determine the value of undertaking a full sys- 
tematic review; (3) to summarise and disseminate 
research findings; and (4) to identify research gaps in the 
existing literature. We identified the first and fourth 
goals as the most relevant for our own research. Arksey 
and O'Malley's framework includes six stages, the sixth 
being optional: (1) identifying the research question, 
which is generally broad in nature; (2) identifying rele- 
vant studies, a process that is as comprehensive as pos- 
sible; (3) study selection, with the establishment of 
inclusion/ exclusion criteria, based on familiarity with the 
literature; (4) charting the data, a stage that includes sift- 
ing, charting, and sorting information according to key 
issues and themes; (5) collating, summarizing, and 
reporting the results, which provides both a descriptive 
and numerical summary of the data and a thematic ana- 
lysis; and (6) a consultation exercise, an additional, par- 
allel step involving key stakeholders to inform and 
validate study findings [1]. While Arksey and O'Malley's 
framework offer the best framework for a scoping study 
to date, we argue that there is room for enhancement. 
Additional file 1 outlines the original framework, Levac 
et al's recommendations, and our recommendations. 

Discussion 

Matching research interests with Arksey and O'Malley's 
framework 

Based on our understanding of Arksey and O'Malley's 
framework, our research team believed a scoping study 
was a methodology that would work well to answer our 
research question: What does the literature tell us about 
the information provided by health care professionals 
and needed by people with colorectal cancers across the 
cancer care continuum? We applied to and received 
funding from a Canadian federal research granting 
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institute under a program specific for scoping studies. 
Receipt of this funding endorsed our belief that this 
methodology would enable us to answer our question. 
However, as we engaged more deeply with our work 
using Arksey and O'Malley's framework and came to 
understand our data, we developed reservations that we 
could claim to understand people's information needs. 
When analyzing our final data set, we were tempted (or 
perhaps eager) to draw conclusions about our interest. 
Instead, we were faced with the dilemma: "what can we 
really say about this data?" It became clear that this 
methodology did not give us the right tools to extract 
the information we sought. We were able to identify 
what issues researchers addressed, but not what infor- 
mation was wanted/needed by people with colorectal 
cancer. In an effort to deal with this tension, we revised 
our research question mid-way through the process into 
a question that we hoped our collected data and the 
Arksey and O'Malley framework could answer: What 
does the literature tell us about which aspects of the in- 
formation provided by healthcare professionals and 
needed by people with colorectal cancer across the cancer 
care continuum have been addressed by researchers? 

Our goals in undertaking a scoping study were to 
examine the extent, range, and nature of research activ- 
ity, and to identify research gaps in the existing literature 
as they relate to information needs of people with colo- 
rectal cancer. We quickly learned that our findings were 
insufficiently specific enough to allow us to fully meet 
our research goals in the way we expected. We were able 
to identify the gaps in what researchers were paying at- 
tention to with regard to information needs of people 
with colorectal cancer and we did understand the extent, 
range, and nature of research activity about the informa- 
tion needs of people with colorectal cancer - there had 
not been much research activity. However, even though 
we reached our goals, we were not able to answer our 
original question because we could not say anything spe- 
cific about those needs. For example: What information 
do people with colorectal cancer say they want and/or 
need? How have these needs been articulated? How and 
when do they want these needs met? What is the reason 
given for these needs? 

Nevertheless, these findings proved useful as they con- 
tributed to our understanding of the state of the litera- 
ture and how we needed to proceed. Importantly, the 
study revealed to us that there is very little literature that 
draws on the perspective of people diagnosed with colo- 
rectal cancer. This inattention to people's voice repre- 
sented a gap in research. It became apparent that 
additional, in-depth research would be required before 
we could draw conclusions. Thus, our research has been 
an essential first step towards developing a research 
agenda where we truly attend to the information needs 



of people with colorectal cancer. Our findings provided 
us with enough data to design a further, more in-depth 
research project, thereby positioning the scoping study 
as an important initial step from which we launched the 
next phase of our research. 

We recognise that we chose the methodology and per- 
haps it was not the best to answer our original question. 
We determined it was important to remain mindful of 
the types of conclusions we could reasonably draw from 
our data gathered through our use of Arksey and 
O'Malley's framework. Thus, we believe that the conver- 
sation about the scoping study methodology that Arksey 
and O'Malley encourage could benefit from guidance 
about the kinds of research questions for which scoping 
studies are most appropriate. Arksey and O'Malley illus- 
trate their framework describing a comparison between 
programs to support carers of people with mental health 
problems. A PUBMED search using the terms scoping 
review or scoping study not systematic review on July 27, 
2012 elicited 101 articles published between July 2007 
and July 2012; 45 of these articles describe studies that 
can be classified as scoping studies, according to Arksey 
and O'Malley's definition, and 34 of them (75%) portray 
some type of comparison such as among interventions, 
programs, and approaches. It seems that this latter kind 
of research is one to which the scoping study method is 
most often applied and is perhaps the most useful. Our 
experience has led us to ponder the appropriateness of a 
scoping study to answer a question that is broader in na- 
ture than the comparison of interventions or programs. 
We were not interested in comparisons of interventions 
and programs, for example. Rather, we hoped to extract 
data that would tell us something concrete about the in- 
formation needs of people with colorectal cancer. As a 
result, we propose that the objective and appropriateness 
of the methodology need to be clearly understood by 
those contemplating its use. However, the boundaries of 
this methodology and the types of research it can best 
support are still to be defined and we urge others to 
contribute to these definitions. It appears that this meth- 
odology may be best suitable to research that examines 
comparisons between interventions or approaches. On 
the other hand, we want to be clear that our research 
was not in vain as we did accomplish two goals laid out 
by Arksey and O'Malley and we garnered data that com- 
pelled us forward into a more in-depth analysis to an- 
swer our initial question in specific detail. Additionally, 
we were able to say useful things about the state of aca- 
demic interest in issues relating to the information needs 
of people with colorectal cancer, the results of which 
have been already published [10]. Because our work 
proved useful in unexpected ways, we encourage others 
to consider a scoping review as a constructive method in 
their broader research process in that it can contribute 
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to the clarification of next steps. By sharing our experi- 
ences with this methodology in this paper, we hope we 
can contribute to the enhancement of Arksey and 
O'Malley's framework. 

Our recommendations in this paper are the results of 
the review of our own research process. We collected 
our team's perspectives through a short survey com- 
prised of three open-ended questions: (1) describe any 
significant challenges/successes that arose from using 
the Arksey O'Malley framework for scoping reviews; (2) 
describe any significant challenges/successes that arose 
from taking part in research with a large, multi- 
disciplinary group; and (3) additional comments. Each 
team member's response was built not only upon mem- 
ory in hindsight, but also upon notes taken throughout 
the research process. At least two key team members 
kept research diaries and a recorder kept minutes 
of each bi-weekly meeting about the challenges, dis- 
cussions, reflections, and actions taken. The survey 
results, the notes, and minutes were collated and ana- 
lyzed. Analysis included identifying themes among 
challenges and successes, identifying relations between 
statements related to any given theme, stating argu- 
ments and supporting them with evidence from the 
materials, making comparisons, and evaluating conclu- 
sions. Results were then linked to specific Arksey and 
O'Malley methodological stages for reporting in this 
paper. 

Enhancing the six steps of Arksey and O'Malley's 
framework: contributing to the conversation 
Arksey and O'Malley's first step: identifying the research 
question 

Our research team initially found identifying an appro- 
priate research question a straightforward task. We gen- 
erated our question on the experience, observations, and 
concerns of our team's health care providers and the 
findings of previous research projects of several team 
members. For example, after a diagnosis of colorectal 
cancer, there are multiple instances where information is 
exchanged and required by people diagnosed with colo- 
rectal cancer, their families, and health care providers. 
These may include diagnostic procedures, receiving the 
diagnosis, treatments and their side effects, and living 
beyond treatment. Our intent was to learn what infor- 
mation people with colorectal cancer needed and/or 
wanted, who or how it was best to provide it to them, 
and when. As per the framework's first and fourth pur- 
poses for conducting a scoping study, we intended to 
examine the extent, range, and nature of research that 
addresses the information needs of people with colorec- 
tal cancer with a view to identifying gaps in the litera- 
ture. With these two goals in mind, we linked our 
purpose to our research question. In an effort to advance 



the Arksey and O'Malley's framework, Levac et al. sug- 
gest the need to clarify and link the purpose and the re- 
search question. Our team did so from the start. 

In keeping with Arksey and O'Malley's recommenda- 
tion to avoid leading with a "highly focused research 
question," [1] we believed our question was sufficiently 
broad in design; it helped us to capture an initial 10,753 
papers. However, once our team delved into the litera- 
ture and began analyzing the data, it became apparent 
that the research question required revision. Levac et al. 
propose researchers define the concepts in their research 
question to clarify the scope of the study. Similarly, 
Arksey and O'Malley suggest that "researchers will want 
to redefine search terms and undertake more sensitive 
searches of the literature" [1]. We held fast to the need 
for a question that was broad in nature. Levac et al's use- 
ful advice was published after we finished collecting our 
data and had started our analysis. By this point in our 
research, we realized we could not say anything substan- 
tive about our research query. We revised our question 
to fit the data we collected instead of revising our ques- 
tion and literature search to get different data that would 
really illicit the specifics of our interest. In hindsight, we 
see Levac et al's suggestion as complementary to Arksey 
and O'Malley's suggestion. 

To help us with this dilemma (and independent of 
Levac's suggestion), we re-reviewed the 239 papers our 
study produced with an additional inclusion criteria - the 
voice of people with a diagnosis of colorectal cancer - and 
identified 64 articles that spoke to the information 
needs of people with colorectal cancer from their point 
of view. The realization of the need to clarify this re- 
search concept came late in our research process. Thus, 
it is only in retrospect that we realized how useful it 
would have been to us earlier. Had we added these in- 
clusion criteria at the outset, we might have had fewer 
articles to analyse but they would have yielded the data 
we were looking for. We recommend that researchers 
follow this advice of Arksey and O'Malley and Levac 
et al. to reduce the need to adjust their question to fit 
the methodology (Additional file 1). 

It is important to reiterate, however, that in spite of 
our challenging experience with Arksey and O'Malley's 
framework, our scoping study was a productive research 
exercise. We learned what has captured the attention 
of researchers and by default, what has not. That we 
narrowed down the original 10,753 papers to a specific 
set of 239 papers indicates the limited amount of litera- 
ture that addresses our subject [10]. Given the pre- 
valence of colorectal cancer, we find this not only 
startling, but useful in directing our future research. We 
are now in the process of further examining the 64 pa- 
pers that reveal the perspective of people with colorec- 
tal cancer. 
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Arksey and O'Malley's second step: identifying relevant 
studies 

This step requires comprehensiveness to be thorough. 
We searched for relevant studies using a general internet 
Google search and several electronic databases, includ- 
ing Google Scholar. In addition, we looked for current 
and closed clinical trials, practice guidelines, meeting ab- 
stracts and dissertations, and searched other sources of 
grey literature using the Grey Matters Checklist created 
by the 2011 Canadian Agency for Drugs and Technology 
in Health. Once we completed these electronic searches, 
we conducted a hand-search of four key journals and 
scanned the reference lists from relevant papers to iden- 
tify other papers that may not have been found in the 
initial search. We also searched the Social Science Cit- 
ation Index for papers that our initial searches may have 
missed [10]. We followed Arksey and O'Malley's rec- 
ommendation to conduct a very broad search since 
comprehensiveness is "the whole point of scoping the 
field" [1]. 

We started with a broad search strategy. Navigating 
and redefining the findings after this initial search added 
another level of complexity to the scoping process. "The 
process is not linear but iterative," Arksey and O'Malley 
emphasize, "requiring researchers to engage with each 
stage in a reflexive way and, where necessary, repeat 
steps to ensure that the literature is covered in a com- 
prehensive way" [1]. Specifically, Arksey and O'Malley 
suggest that researchers fine tune their search of the lit- 
erature. As noted above, we neglected to do this step 
and, in hindsight, realize how useful it would have been. 
We agree that flexibility and comprehensive searches are 
important to successful scoping studies. 

However, a fine balance must be struck between the 
laborious nature of study identification and the need for 
comprehensiveness on the one hand, with the need to 
complete a scoping study in a timely fashion, on the 
other. Levac et al. and others recognize this issue [2,4]. 
While Levac et al. recommend assembling a "suitable 
team with content and methodological expertise" [8] to 
identify studies, we recommend assembling a small suit- 
able team to perform this task (Additional file 1). We 
were a large team and we quickly found it counterpro- 
ductive for every team member to be actively involved at 
the beginning of the research process; with too many 
people, there may be the introduction of too many inter- 
pretations, thereby leading to inconsistent search results. 
In order to maintain consistency and control, we tasked 
our librarian team member with the design and execu- 
tion of search strategies. Navigating this many references 
took considerable time and skill. Although our search 
produced such a significant number of references, it left 
our research team confident that we had scoped the field 
as comprehensively as possible. Additionally, the use of 



an online citation management software proved to be of 
immeasurable benefit. It allowed us to organize and 
cross-check our data and remove duplicates. It also 
allowed all those involved in the charting steps of the 
study easy access to the papers. 

Arksey and O'Malley's third step: study selection 

Once we identified the relevant literature, our team 
established the exclusion and inclusion criteria to apply 
to the papers. The librarian and research assistant on 
our team applied these criteria to the paper abstracts. 
We included studies that mentioned in any way the in- 
formation needs of people with a diagnosis of colorectal 
cancer, their families and caregivers, or the information 
sources used by them. We carefully set parameters 
around studies for exclusion. For example, we excluded 
studies related to screening, prevention, genetics, and 
studies that included other types of cancer in addition to 
colorectal cancer. By applying the exclusion criteria to 
the abstracts, we reduced the numbers of papers that 
needed to be read in detail to 869 papers. 

Given our large team, we made an adjustment to 
Levac et al's suggestion of engaging two reviewers to 
apply the inclusion/exclusion criteria to the papers. Spe- 
cifically, we took a three-tiered approach to narrow 
down and cross-check our papers. First, we divided 
the research team into six teams of two people each. 
Each team was assigned one sixth of the 869 articles to 
review. Both members of each team independently 
reviewed the full text of each article to determine its eli- 
gibility for our study according to the inclusion and ex- 
clusion criteria. Second, team members then compared 
their results to reach an agreement about which articles 
they recommended to include or exclude. Third, if they 
could not reach an agreement, the research assistant or 
the librarian acted as a third reviewer. After concluding 
this process, a total of 239 papers met the inclusion cri- 
teria and were included in our study. We made this ad- 
justment to Arksey and O'Malley's framework due to the 
size of our research team. We wanted to give each of 
our twelve team members exposure to the data and to 
reduce the workload by dividing up the work tasks. 
Levac et al. reached a similar conclusion about how to 
adjust the application of the inclusion/exclusion criteria, 
confirming our own decision to modify this portion of 
the framework (Additional file 1). 

Levac et al. and others [2,3,5] are concerned about 
Arksey and O'Malley's framework's inability to provide 
for an assessment of the quality of the literature. Because 
our research team is using our scoping study as the basis 
for our next stage of research, we did not focus too 
much on this concern (although it did concern us from 
the outset). However, in retrospect, we believe assessing 
for quality is a necessary component of scoping studies 
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if they are to provide research that in itself can be dis- 
seminated to others in a way that is useful to practice or 
policymaking and for future researchers. We strongly 
recommend that assessing for quality be factored into 
Arksey and O'Malley's framework and suggest adding 
these criteria to the selection of studies to be charted. 
Some suggested methods for assessing quality might in- 
clude, for example, using validated instruments. 

Arksey and O'Malley's fourth step: charting the data 

To organize data, Arksey and O'Malley recommend 
charting and sorting data according to key themes and 
issues. They suggest that data charted should include a 
mixture of general information about the study and spe- 
cific information related to the study question. Arksey 
and O'Malley's own scoping work, an examination of the 
literature about services or programs for a particular 
population, charted seven pieces of information on each 
article (general information: citation data, methodology, 
aims of the study, outcome measures, important results; 
specific information: intervention type, study popula- 
tion). Researchers can determine what categories to 
chart for their particular study. We opted to adapt a 
charting format from Rutten et al. [11], a systematic re- 
view investigating the information needs of people with 
cancer. The categories and subcategories used by Rutten 
et al. fit well with our research question as our goal was 
similar but focused on a specific type of cancer, colorec- 
tal. Because our team consisted of health care providers 
with extensive clinical experience, we were able to an- 
ticipate the complexity of patient information needs 
prior to conducting our scoping study. To this end, we 
expanded and modified the set of subcategories related 
specifically to information needs from the original 64 
(from Rutten et al.) to a total of 82. Using such a com- 
prehensive chart proved both a strength and a limitation 
of our research. It was a strength because our final set of 
data was rich. We collected more data than we had 
room to report in a single manuscript. On the other 
hand, it was time consuming to chart with such a large 
number of categories. Some of our team members who 
were busy, practicing health care providers found the de- 
tailed nature of the charting process challenging because 
it took time, first, to engage with the information we 
aimed to capture and, second, to actually capture that 
information. 

Levac et al. suggest that research team members, inde- 
pendent of one another, chart the "first five to ten stud- 
ies using the data-charting form and meet to determine 
whether their approach to data extraction is consistent 
with the research question and purpose" [8]. We under- 
took a similar step in that we conducted a trial charting 
exercise followed by a group consultation. This step 
added value by providing our team an opportunity to 



reach consensus about our data collection process and 
to include additional categories relevant to our research 
inquiry. It improved the quality and applicability of the 
chart used as well as consistency. We recommend re- 
searchers consider this step to ensure the data they col- 
lect are as rich as possible. 

Using our revised Rutten chart, we repeated a similar 
process as in step three with our six teams of two mem- 
bers each, where each team member worked independ- 
ently to chart the included articles and team members 
then compared their results to agree upon the charting 
of each article. However, we changed the membership of 
each team in an effort to ensure consistency in interpret- 
ation and validity in our results. The research assistant 
read and charted all 239 articles. All three reviewers 
discussed any discrepancies and agreed upon a final in- 
terpretation. Having one person read and chart each art- 
icle ensured inter-reviewer reliability throughout the 
process and contributed to our confidence in the 
consistency of our data results; this process was an im- 
portant and valuable part of our research. During this 
process, effective communication to maintain a clear 
framework and consistent charting methods cannot be 
underestimated, particularly for a large team. 

Arksey and O'Malley's framework recommended using 
Excel to chart each paper, but they did not advise their 
readers about a key feature of data management: the im- 
portance of assigning each paper a unique identifying 
number. We did this task early on as a way to conveni- 
ently and clearly track those articles we excluded and 
those we included. We also recommend this step be- 
cause it provides a useful way for a large team to talk 
about a particular article without having to cite full ref- 
erences or get confused by the names of researchers 
who have published multiple papers. 

It was during the charting step that we began to 
realize our inability to answer our research question. We 
charted that an article mentioned a particular informa- 
tion need, yet we could not say anything about that 
need. For example, did people with colorectal cancer say 
they wanted this information or did the authors note it 
as information given to people with colorectal cancer, 
and it was unconnected to the people receiving it? 

Arksey and O'Malley's fifth step: collating, summarizing, 
and reporting the results 

When reporting the data in our study, it was imperative 
that we used a consistent, clear approach. Following Arksey 
and O'Malley's framework, we charted the extent, nature, 
and distribution of the included articles. As previously 
mentioned, we made additions to the Rutten et al. charting 
categories to better suit the complex information needs of 
people with colorectal cancer. Our goal with the 15 cat- 
egories (ten broadly defined categories of information needs 
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and five broad information sources categories) was to 
gather high-level data about the types of information 
needs, while we hoped the 101 sub-categories (82 sub- 
categories of information needs and 19 sub-categories of 
information sources) would gather micro-level, specific 
data on these needs. We searched for this range of data in 
order to satisfy the varied interests of each of our inter- 
professional team members. These team members desired 
this level of detail in order to make sense of the data in re- 
lation to their own professional practice. This proved to 
be a real strength to our approach to scoping studies and 
affirmed the value of including health care providers on 
our research team. Our ability to capture a complexity of 
data did not necessarily get us closer to answering our ori- 
ginal research question, but it did provide us with an indi- 
cation of the types of information needs being discussed 
by researchers. 

We made sense of our complex data with a focus on our 
research question. Once we gathered all of our data and 
produced some preliminary graphs portraying number of 
mentions and specific needs, we established a smaller 
working group to make meaning out of the data and to 
make choices about the data on which we should focus. 
By this point, we knew we were not answering our initial 
question. We then modified our question and followed 
Arksey and O'Malley's suggestion to prioritize certain as- 
pects of the literature according to implications for future 
research and what was most notable given our experience. 
With our research question in mind, we first quantified 
the data and produced graphs and charts to represent 
these numbers. We then did a thematic analysis that 
resulted in organizing our data into three overarching 
themes: the information needs of people with colorectal 
cancer, the sources of this information, and the timing of 
these information needs. Our steps mirrored Levac et al's 
recommendation to add a stage resembling qualitative 
data analytical techniques, or a thematic analysis. We en- 
dorse Levac et al's suggestion (Additional file 1). 

We published our findings using a combination of ta- 
bles with descriptions according to our themes [10]. This 
approach proved useful to us in our analysis and allowed 
us to clearly link our findings with our research goals: to 
examine the extent of the literature and to identify gaps. 
Our data also pointed very clearly to the need for future 
research to pay attention to the need for information as 
articulated by people with colorectal cancer. We do not 
offer a new recommendation in this regard, but support 
Levac et al. in their suggestions for this stage of the 
scoping study framework. 

Arksey O'Malley sixth step: optional stage, consultation 
exercise 

Arksey and O'Malley suggest an optional consultation 
stage with stakeholders, whereas Levac et al. suggest it 



should be a requirement. Based on our experience, we 
agree with Levac et al. (Additional file 1) but with room 
for interpretation of how that consultation is achieved. 
In our study, there were two groups of stakeholders: 
health care providers providing care to people with colo- 
rectal cancer and people with colorectal cancer them- 
selves. Our research team included a significant number 
of health care providers. Their participation throughout 
the entire research process served the purpose of con- 
stant consultation and was a real strength of our inter- 
professional approach. We were able to extract multi- 
faceted perspectives in the literature and our findings 
throughout the entire research process rather than leav- 
ing it to the end of the project. As a result, we were able 
to revise the charting categories early on so it more ac- 
curately reflected people's experience than if we had just 
borrowed one. The health care providers drew upon their 
experience to identify the gaps in the categories being un/ 
attended to by researchers. We originally intended to con- 
sult with the second group of stakeholders, people diag- 
nosed with cancer. However, we omitted that step after 
realizing that our study did not tell us anything about their 
information needs from their perspective because it gave 
us nothing substantive on which to seek their consult- 
ation. With Arksey and O'Malley's framework, consult- 
ation only works if the results are germane to the group 
that researchers wish to consult. Moving forward with fu- 
ture research, we have built this consultation process into 
our current research with the 64 articles. We see it as a 
fundamental step to determine how what the literature 
says resonates with people with colorectal cancer. 

Levac et al. and Brien et al. [5,8] discuss the importance 
of knowledge translation with stakeholders in the field, a 
perspective we share. In fact, Grant and Booth state that 
"scoping reviews are able to inform policymakers as to 
whether a full systematic review is needed" [4]. Although 
the methodology did not enable us to address our specific 
research question, the scoping study exercise did demon- 
strate that a further study is needed and it pointed out 
gaps in the literature. In fact, our experience led us to 
believe that scoping studies may be a useful step - perhaps 
even a necessary pre-requisite - toward a research project 
that allows for a deeper analysis, such as a knowledge syn- 
thesis methodology [12]. Indeed, our study has been valu- 
able in part because it opened up questions to guide 
future research from which we will have findings we can 
share with stakeholders, including policymakers and 
health care providers to inform their practice. 

Inviting suitable stakeholders to be part of the research 
team is one way to incorporate the consultation Levac 
et al. say is necessary. We recommend researchers con- 
sider this decision when possible as it makes the entire 
research process and, thus, the findings, rich. That being 
said, not all stakeholders will be appropriate to be 
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involved on the research team and researchers must 
make this decision carefully. For example, previous ex- 
perience with research is not necessarily a requirement 
but experience in a critical analysis of relevant literature 
is necessary. Effective team work skills are imperative, 
especially when large groups are assembled. Time avail- 
ability to participate in all stages of the project is another 
important point to be considered. 

A large, inter-professional team using Arksey and 
O'Malley's framework: strengths and challenges 

Our contribution to the conversation on scoping studies 
relates to our experience as a large, inter-professional 
team using Arksey and O'Malley's framework. Levac 
et al. and others point to a multidisciplinary team as a 
benefit for scoping studies [2,13]. According to Ander- 
son et al., a multidisciplinary team provides the required 
"expertise to map a subject" that is "not necessarily al- 
ways found in one researcher" [2]. Our research team 
was not only multidisciplinary, but inter-professional in 
its composition. A multidisciplinary team is academic in 
character because it joins together members trained in 
different disciplines, whereas an inter-professional team 
is professional in character because it joins together 
members trained in program areas of clinical practice, 
plus academic researchers [14]. It is important to note 
that the challenges we experienced are not unique to 
scoping studies, but are faced commonly by all research 
projects conducted by large, inter-professional teams. 
However, critical discussions about inter-professional 
teams are absent from the current literature about scop- 
ing study methods. 

The large, inter-professional character of our team 
brought both strengths and challenges to Arksey and 
O'Malley's framework. On one hand, it was a strength 
because our team collectively brought a breadth and 
depth of clinical and research knowledge absent in a 
team comprised only of academic researchers. It also 
brought time efficiencies. Our team worked efficiently 
when it came to assigning and completing work tasks. 
We strategically divided up work tasks, which often 
meant a lighter workload for each individual team mem- 
ber. This type of time efficiency was especially evident at 
the inclusion/exclusion and charting stages of Arksey 
and O'Malley's framework. The make up of our team 
also added to the rigor of the research process; twelve 
team members from diverse backgrounds provided many 
perspectives. 

On the other hand, the large, inter-professional char- 
acter of our team had some challenges relating to con- 
sensus, research experience, and resource limitations. 
The greater the number of team members, the greater 
the number of perspectives. At certain points, productiv- 
ity decreased. Although such a team produced rich 



analysis, it was challenging to reach consensus because 
there were occasional competing perspectives. At times, 
we found it difficult to arrive at a common language or 
vocabulary for discussion and had to re-double our ef- 
forts during the research process to ensure consistency. 

Finally, as with any research project, there is always a 
need to balance finite resources of time and money. With 
an inter-professional team, this need is greater because 
there are more competing interests, more professional 
schedules to coordinate, and more communication efforts 
required to explain, explore, and reach consensus about 
ideas. We found it helpful to designate one person or 
a small section of the team to take the lead at various 
stages of the research process to move the project forward. 
While we have noted at several steps of the process that 
communication and regular meetings are important, we 
also acknowledge that it may be challenging to organize 
meetings for so many team members and to ensure that 
all team members consistently produce deliverables at 
agreed upon deadlines. We recommend, however, that 
large, inter-professional teams pay special attention to this 
issue as communication is key to success. 

Summary: lessons learned 

Some authors have expressed their concerns about Arksey 
and O'Malley's framework's inability to provide for an as- 
sessment of the quality of the literature [2,3,5,8]. We be- 
lieve assessing for quality is a necessary component of 
scoping studies. The assessment itself is a significant task 
and should be performed using validated instruments. 
This recommendation does not stand alone, however. We 
propose that the recommendations of Levac et al. to clar- 
ify the concepts in the research question and Arksey and 
O'Malley's suggestion to redefine search terms will ensure 
the assessment of quality actually proves useful. To this 
end, the definition and purposes of scoping studies must 
be made in tandem with clarifying the methodological 
framework; no one can be done without the others. 

We endorse Arksey and O'Malley's purposes when 
taking into consideration Levac et al. and our own rec- 
ommendations. As for the definition used by Arksey and 
O'Malley, we would remove the term "rapidly" and re- 
place it with the need for scoping studies to be done 
thoroughly and thoughtfully. To be clear, scoping studies 
take time. Second, we think that adding a quality assess- 
ment step would alter the definition. 

With a more detailed methodological framework 
according to the recommendations of Levac et al., our 
own, plus purposes that are more attainable, we suggest 
a definition of scoping studies as follows: Scoping studies 
aim to map the literature on a particular topic or re- 
search area and provide an opportunity to identify key 
concepts; gaps in the research; and types and sources of 
evidence to inform practice, policymaking and research. 
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Creating a large, inter-professional team to engage in a 
scoping study offers tangible benefits to the research 
process. If using a revised framework set out by Arksey 
and O'Malley's using Levac et al. and our recommenda- 
tions, major tasks can be fairly divided without com- 
promising the consistency of analysis. Additionally, each 
inter-disciplinary team member adds richness to the 
analysis as he/she brings his/her perspective to the re- 
search table. 

A large, inter-professional team including health care 
providers allowed us to anticipate the complexity of in- 
formation needs. Including suitable stakeholders on the 
research team builds in the consultation stage through- 
out and adds depth to all stages of the research process. 

It is important that researchers fully understand the 
methodology's boundaries and its inherent methods. It 
would be beneficial to include someone on the research 
team who is experienced with scoping studies. Most im- 
portant is the work involved of matching the methodology 
with research interests and attending to suggestions to 
clarify concepts within the research question and/or focus 
the literature search as necessary to ensure researchers do 
not lose sight of their interests. 

Additional file 



Additional file 1: Table SI. Comparative recommendations to enhance 
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